Papers with zero-shot tasks
RespiraMFM: A Multimodal Foundation Model with Contrastive Audio-Language Alignment for Respiratory Disease Identification (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models for respiratory diseases rely on audio inputs, but they lack generalizability and diagnostic precision. |
| Approach: | They propose a multimodal foundation model that integrates respiratory sounds with medical history and symptoms to enhance diagnostic accuracy and disease detection capabilities. |
| Outcome: | The proposed model improves AUROC and zero-shot tasks across five respiratory diseases using real-world datasets. |
In-Context Demonstration Selection with Cross Entropy Difference (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) can use in-context demonstrations to improve performance on zero-shot tasks. |
| Approach: | They propose a cross-entropy difference method for selecting in-context demonstrations that uses parameter efficient finetuning to train small models on training data. |
| Outcome: | The proposed method outperforms baseline selection methods on a mix-domain dataset and shows that the effectiveness of in-context demonstrations negatively correlates with the perplexity of the test example. |
Consistency by Agreement in Zero-Shot Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | In this paper, we focus on zero-shot generalization—a challenging setup that tests models on translation directions they have not been optimized for at training time. |
| Approach: | They propose a method that allows for a consistent agreement-based training method that encourages the model to produce equivalent translations of parallel sentences in auxiliary languages. |
| Outcome: | The proposed model improves on public zero-shot translation benchmarks without loss of performance on supervised translation directions. |
Anti-LM Decoding for Zero-shot In-context Machine Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to zero-shot learning with large language models are poorly calibrated for zero-shoot tasks. |
| Approach: | They propose a contrastive decoding objective with a decay factor to address in-context bias . they conduct experiments on 3 model types and sizes, 3 language directions, and beam search . |
| Outcome: | The proposed method outperforms state-of-the-art decoding objectives with 20 BLEU points improvement from the default objective in some settings. |
Wav2Prompt: End-to-End Speech Prompt Learning and Task-based Fine-tuning for Text-based LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Text-based large language models (LLMs) can be applied to a wide range of tasks without being explicitly trained. |
| Approach: | They propose a method which integrates spoken input with a text-based large language model (LLM) it takes LLM token embeddings as training targets and utilises a continuous integrate-and-fire mechanism for explicit speech-text alignment. |
| Outcome: | The proposed model can be applied to speech translation, speech understanding and spoken-query-based question answering tasks. |
HYDEN: Hyperbolic Density Representations for Medical Images and Reports (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for visual semantic representation learning struggle to address semantic uncertainty, especially in the medical domain. |
| Approach: | They propose a hyperbolic density embedding based image-text representation learning approach tailored for specific medical domain data. |
| Outcome: | The proposed method performs better than baseline methods on zero-shot tasks and fine-tuning tasks on different datasets. |
Q-Mamba: Towards more efficient Mamba models via post-training quantization (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that Mamba architectures have room for further optimization in linear projections and state caches. |
| Approach: | They propose a decoupled scale quantization scheme to mitigate outliers in states and channels by applying separate quantization scales. |
| Outcome: | The proposed method reduces memory consumption by 50% across various quantization settings, model sizes, and generation and zero-shot tasks. |
MoE-I2: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition (2024.findings-emnlp)
Copied to clipboard
Cheng Yang, Yang Sui, Jinqi Xiao, Lingyi Huang, Yu Gong, Yuanlin Duan, Wenqi Jia, Miao Yin, Yu Cheng, Bo Yuan
| Challenge: | emergence of Mixture of Experts (MoE) LLMs has significantly advanced the development of language models. |
| Approach: | They propose a two-stage compression method tailored for Mixture of Experts to reduce the model size and decrease the computational cost. |
| Outcome: | The proposed method reduces model size and improves inference efficiency while maintaining performance in various zero-shot tasks. |
Can Medical Vision-Language Pre-training Succeed with Purely Synthetic Data? (2025.findings-acl)
Copied to clipboard
Che Liu, Zhongwei Wan, Haozhe Wang, Yinda Chen, Talha Qaiser, Chen Jin, Nikolay Burlutskiy, Fariba Yousefi, Rossella Arcucci
| Challenge: | Medical Vision-Language Pretraining (MedVLP) models typically require large-scale datasets with paired, high-quality image-text data. |
| Approach: | They propose to generate large-scale synthetic image-text pairs using off-the-shelf generative models . they propose to isolate model and training settings, focusing entirely from the data perspective. |
| Outcome: | The proposed pipeline outperforms models trained on real data by 3.8% on averaged AUC on zero-shot classification tasks. |
BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models have driven major progress in NLP, but memory and compute requirements hinder practical deployment. |
| Approach: | They propose a framework that preserves high accuracy while achieving 1-bit weight quantization . the orthogonal-kronecker transformation learns an orthogonale mapping via EM minimization - a new approach to quantization is proposed . |
| Outcome: | The proposed framework achieves 1-bit weight quantization with low activations with low-bit activations. |
Discriminating Form and Meaning in Multilingual Models with Minimal-Pair ABX Tasks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that multilingual models encode languagespecific information and language-agnostic features, but the nature and interaction of these representations is not fully understood. |
| Approach: | They propose a set of training-free ABX-style discrimination tasks to evaluate how multilingual language models represent language identity (form) and semantic content (meaning). |
| Outcome: | The proposed tasks show that language discrimination declines over training and strengthens over time and stabilizes in deeper layers. |
Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Quantization enables efficient deployment of large language models in resource-constrained environments . but impact on truthfulness remains largely unexplored . |
| Approach: | They propose a framework to assess the truthfulness of quantized large language models . they find quantized models retain internally truthful representations but produce false outputs . |
| Outcome: | The framework assesses the truthfulness of quantized models across three dimensions . it finds that quantized model models retain internally truthful representations but are more susceptible to false outputs . |